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ABSTRACT 


Maculopathy and retinitis pigmentosa (rp) are two vision defects 
which render the afflicted person with impaired ability to read and 
recognize visual patterns. For some time there has been interest and 
work on the use of image remapping techniques to provide a visual aid 
for individuals with these impairments. The basic concept is to remap 
an image according to some mathematical transformation such that the 
image is warped around a maculopathic defect (scotoma) or within the rp 
foveal region of retinal sensitivity. NASA/JSC has been pursuing this 
research using angle invariant transformations with testing of the 
resulting remapping using subjects and facilities of the University of 
Houston, College of Optometry. Testing is facilitated by use of a 
hardware device, the Programmable Remapper, to provide the remapping of 
video images. 

This report presents the results of studies of alternative 
remapping transformations with the objective of improving subject 
reading rates and pattern recognition. In particular a form of 
conformal transformation was developed which provides for a smooth 
warping of an image around a scotoma. In such a case it is shown that 
distortion of characters and lines of characters is minimized which 
should lead to enhanced character recognition. In addition studies were 
made of alternative transformations which, although not conformal, 
provide for similar low character distortion remapping. 

A second, non-conformal transformation was studied for remapping 
of images to aid rp impairments. In this case a transformation was 
investigated which allows remapping of a vision field into a circular 
area representing the foveal retina region. The size and spatial 
representation of the image are selectable. It is shown that parametric 
adjustments allow for a wide variation of how a visual field is 
presented to the sensitive retina. 

This study also presents some preliminary considerations of how a 
prosthetic device could be implemented in a practical sense, vis-a-vis, 
size, weight and portability. 
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INTRODUCTION 


Maculopathy and retinitis pigmentosa (rp) are two forme of human 
vision defects which are characterized by partial loss of ocular 
function. In the case of maculopathy the dysfunction is in the foveal 
region of the retina and thus the area of greatest resolving power. The 
resulting scotoma or "blind spot" leaves individuals so afflicted with 
an impaired ability to read and to recognize patterns since both 
functions involve high resolution visual imagery. They learn to 
partially compensate by fixating on a spot removed from the focus of 
attention such that the image falls on the most optimum peripheral 
retina. Figure 1 illustrates how a sample of written material might 
appear to a maculopathic individual where the eye is "focused" upon the 
region directly under the scotoma blocked text. If fixation is made on 
the center of the circular area the difficulty of character recognition 
using images falling on peripheral retina can be noted. 
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The case of retinitis pigmentosa is the inverse of maculopathy in 
that it is the peripheral regions of the retina that lose sensitivity 
while the foveal region remains functional. Individuals afflicted with 
this "tunnel vision" condition have difficulty in navigation and 
dexterous activity since loss of peripheral vision disturbs the 
awareness of local surroundings. Sensitivity to motion detection is 
also impaired. Afflicted individuals learn to compensate by swinging 
the head during ambulation to sweep the field of view through the 
sensitive retinal region. 

As part of space activity related research NASA Johnson Space 
Center has developed considerable expertise in the theory and techniques 
of visual image warping. The Programmable Remapper ^ (PR) is a hardware 
device developed by NASA/ JSC which is capable of providing specified 
warping of an input TV image to an output monitor display at better than 
30 frames per second. Considerable research has been conducted on how 
image warping may be applied as a prosthesis for individuals afflicted 
with maculopathy or rp. The basic concept is to warp the image such 
that a portion of the lost data is redistributed to remaining sensitive 
regions of the retina^"*. Research thus far has involved remapping of 
textual material in such a way as to warp the image partially or totally 
exterior to an assumed scotoma. The warping transformation employed in 
these initial investigations is an angle invariant shift of the image 
along radial lines from the center of the scotoma. Figure 2 illustrates 
the appearance of warped text for which approximately 80% of the image 
has been remapped outside of an assumed scotoma. Experimental testing 
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W as accomplished by using the PR in conjunction with an eye tracker to 
provide a warped image for subjects with simulated maculopathy • The 
results suggest some improvement in reading capacity 5 . 
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Figure 2. Radial warping of text. 


The particular radial transformation employed in the testing and 
shown in Figure 2 results in considerable distortion of the text 
characters. This may adversely effect the results of reading rate 
measurement since, although the image has been warped to a sensitive 
region of the retina, character distortion hinders recognition. 

This paper presents results of two aspects of continued work on the 
application of warping strategies toward development of a prosthesis for 
those with maculopathic or rp afflictions. The first aspect involves a 
study of alternative warping transformations and the second focuses on 
issues related to the practical and feasible implementation of a 
prosthesis. 


Transformation Formulation 


In general a visual image to be remapped conaiets of a rectangular 
array of pixals, whara each pixal ia represented by a two dimensional 
coordinate and has a "value" which describes the attributes of the 
pixel. Attributes include such features as intensity and color. The 
transformation process specifies how a pixel located in the image input 
plane at (x,y) will be re-mapped to the image output plane at location 
(u,v). This transformation is specified by equations relating (x,y) to 
<u,v) , 

u * f (x,y) (1) 

v « g(x,y) (2) 



Input plane Output plane 

Figure 3. The transformation process 
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Whereas the process of Equations (1) and (2) is analytically 
complete it fails to consider the discrete nature of the pixel lated 
image. If the Jacobian of the transformation is greater than unity in 
some region then a single pixel in the input plane may transform into 
several output plane pixels. Of course if the Jacobian is less than 
unity the converse may occur wherein several input plane pixels 
transform into a single output plane pixel. In these cases 
interpolation processes must be used to relate un-mapped input and 
output pixels and the attributes of the input and output plane pixels 
must be modified to retain image features such as color and contrast. 

The Programmable Remapper described earlier includes pixel 
interpolation and attribute adjustments so that only the analytical 
features of the transformation need to be developed. 

In order to statically test the transformations studied in this 
work the test text of Figure 1 was employed. The transformation 
equations were used to recast this image so that the effect of the 
warping could be examined. Since the objective was to study the effect 
of transformations on character shape and orientation, only very 
elementary interpolation and fill-in routines were employed. Therefore 
the appearance of the test text after warping as shown in this report is 
is only partially filled and is not to be interpreted as the final 
representation for actual patient testing. 


HACULOPATHIC TRANSFORMATIONS 

Study of alternative transformations for maculopathy compensation 
was confined to those which appeared to be most suitable for enhanced 
reading skills as opposed to facial recognition. It is not at all clear 
that the optimum transformation for the first would also be optimum for 
the second. This choice was made since experimental studies are 
presently concentrated on measurement of reading enhancement via image 
warping. 

It is assumed that an optimum text warping transformation would be 
one which: 

(1) displaces the text image blocked by the scotoma to sensitive 
retina just outside the scotoma, 

(2) minimizes distortion of the shape of the text characters, and 

(3) maximizes preservation of the textual flow of characters as 
lines are warped around the scotoma. 

Of course the true test of a transformation is in its efficacy in 
improvement of maculopathic reading rate. 

A Conformal Transformation - sin^fz) 

A conformal transformation is one which preserves angles between 
line segments through the transformation. This means that the basic 
orientational features of characters and lines consisting of characters 
would be preserved. This is consistent with the optimum transformation 
objectives. The conformal transformation employed was a two step, 
complex arc sin /complex sin process. 

The first step is to perform a transformation of the form, 

z * - sin -1 (z) (3) 


15-5 



where z* * x ' + jy* is in an intermediate image plane 

and z * x + jy is formed from the input image plane 

An offset is now added to y', 

y * * * y* + c (4) 

so that z ,f * x* + jy' 1 * 

The reverse transformation is used to construct the output image plane, 

w « sin(z'*) * u + jv (5) 

It is easy to show that this transformation creates an elliptical void 
about the origin of the (u,v) plane with dimensions shown in Figure 4. 
Essentially the transformation expands the line from (0,-1) to (0,+l) in 
the input image plane into the indicated ellipsoidal shape* Thus images 
in the input plane will be warped into the output image plane such that 
all input image pixels are remapped into the space surrounding the 
elliptical void. 



Figure 4. Conformal remapping by arcsin/sin 


Since unity points play an important role in this remapping, proper 
normalization of the input image must be included. To test this 
transformation for maculopathy a circular scotoma of radius r m is 
assumed, as shown in Figure 1. The input image is then normalized to 
some radius, r n , about the scotoma site, to define the unity points. If 
the entire scotoma 1 area is to be remapped then the origin in the input 
image plane must map to the edge of the scotoma on the output image 
plane* In general one may wish to remap some fraction, a, of the 
scotoma blocked region of the input image to unblocked regions of the 
output image* This defines the value of the transformation 
displacement, c, through the relation, 

■inh(c) - a(r./r n ) (6) 

Thus two parameters determine the features of the remapping. The 
fraction of image recovery, a, and the normalization radius, r n . 
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The transformation was teBted using a * 0.8. Figure 5 shows the 
result for the case r B /r n * 0.5, which means that the unity points lie 
outBide of the scotoma. 
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Figure 5. Conformal transformation, 80% and r 8 /r n * 0.5 


Notice the way the characters flow around the scotomal area while 
retaining their effective shape and orientation. Also the process of 
displacement in the transformed domain results in a magnification. 
Figure 6 shows the remapping which results from a remapping with r 8 /r n 
1.0 and Figure 7 for r 8 /r n * 1.5. Note the increased magnification in 
each case. 
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Figure 6. 


Conformal transformation. 
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The above examples assume that the fixation point of the observer 
is between lines of text. The effect of scanning the fixation point 
down across the lines of characters can be seen in Figure 8. 
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Figure 8. Effect of changing fixation point on warping 

( 80%, r s /r n - 0.5) 


Experimental study of the efficacy of this conformal transformation 
on reading enhancement is presently underway at the University of 
Houston, College of Optometry. 

Hyperbolic Secant Transformation 

Another transformation studied which has many of the desired 
features for the optimum transformation is defined by the following 
equations: 

u * x (7) 

v * y + a(sech(bx)) (8) 

The effect of this transformation is such that the y space of the input 
plane is warped such that an approximately ellipsoidal void, similar to 
the conformal, is produced. Figure 9 shows how the test text is warped 
by this transformation. 
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Figure 9. Secant transformation 

The x space is unchanged. Thus the transformation pushes 
characters up but does not totally preserve the shape of letters. 

Unlike the confromal transformation there is no magnification associated 
with the warping, although this can be added. Variation of parameters 
allows for changing the roll-off of character lines and the vertical 
extent of the warping. Experimental studies of this transformation will 
be conducted at the College of Optometry of the University of Houston. 
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RETINITIS PIGMENTOSA TRANSFORMATIONS 


Transformations for retinitis pigmentosa (rp) are required to remap 
the input plane into a smaller field of view centered on the fovea. 

This kind of transformation leads one to consider transformations which 
map a Cartesian system into the unit circle as suggested in Figure 10. 



During this project a radial transformation was studied which maps the 
entire world view into a circular region of variable radius. By 
variation of transform parameters it is possible to vary not only the 
radius of the output plane image but also the region of the input plane 
which appears relatively undistorted in the output plane. The 
transformation equations are given by: 


r * [x 2 + y 2 ] 1 / 2 


u * [1 - e“9 r ]x/r 

(10) 

v * [1 - e~9 r ]y/r 

(11) 


Figures 11 , 12 and 13 show three samples of how the test text would 
appear following transformations given by Equations (10) and (11). Note 
that the entire field of Figure 1, without the scotoma overlay of 
course , is included in these image views. The original image has been 
compressed toward the edges in a circularly symmetric way. 
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Figure 11. RP image for wide vision 
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Figure 12. RP image for intermediate vision 





Figure 12. RP image for narrow vision 


Although these show the effect of the remapping it should be noted 
that the real test of rp transformations lies in pattern recognition. 

The next phase of this work will be to implement transformations such as 
that presented here in a portable viewing system. This will study the 
efficacy of the system for enabling a person afflicted with rp to 
navigate, locate and retrieve objects. It will be necessary to be able 
to switch the rp correction system to provide various views such as for 
enhanced near-field vision or far-field vision. 

IMPLEMENTATION CONSIDERATIONS 

There were two issues explored during the project period with respect 
to implementation of a practical prosthesis using transformations. The 
motivation for study of this problem is the need for a portable, "heads 
up" system for practical use by a visually impaired individual. 

Although the Remapper is able to provide up to 30 frames per second, it 
consist of a large, heavy computer system with associated power 
supplies, monitors and so forth. The need is for a process by which a 
system could be head mounted, with associated video camera(s) and 
miniature display (s). 

The first issue was to consider the construction of a sensor at the 
front end of the system which could perform part of the transformation 
in a hardware sense. The concept here is that variation of the size and 
physical geometry of the sensor pixel array could effect part of the 
required transformation 6 . Discussions were held with industrial 
representatives from Texas Instruments regarding the feasibility of 
building such a sensor. It appears that there are significant obstacles 
to such a device although similar sensors have been fabricated. This is 
an area for further study. 

The second study regarded the possibility of implementation of an 
effective sensor using optical, distorted fiber bundle technology. In 
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this case consideration was given to the use of a fiber-optic bundle 
which has been tapered. Such a device is commonly employed to provide a 
magnified or reduced image. In the case of a low vision prosthesis for 
maculopathy the die forming the original bundle would have a central 
hollow core as shown in Figure 14. When heated and drawn the core would 
normally reduce in proportion to the fiber bundle major radius. It may 
be possible however to draw down the heated bundle such that the core 
radius reduces to zero as the whole bundle is drawn. This would result 
in a structure similar to that in Figure 14. It is clear that if an 
image is presented to the reduced area surface it will be projected onto 
the larger radius surface with a void in the central region, where the 
scotoma would be located. In this way the image is warped around the 
scotoma. 



Figure 14. Taper glass fiber sensor 

Whereas there is some technical reason to believe this process would 
work, the resulting remapping does not have good features with respect 
to providing a readable image. Further consideration will be given to 
this process and possible modifications to provide for useful 
transformations • 
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